Field of Invention
Aspects of this invention are related to endoscopic imaging and are more particularly related to nonwhite light used for general illumination in a teleoperated surgical system.
Related Art
The da Vinci® Surgical System, commercialized by Intuitive Surgical, Inc., Sunnyvale, Calif., is a minimally invasive teleoperated surgical system that offers patients many benefits, such as reduced trauma to the body, faster recovery and shorter hospital stay. One feature of the da Vinci® Surgical System is a capability to provide two-channel (i.e., left and right) video capture and display of visible images to provide stereoscopic viewing for the surgeon.
Such electronic stereoscopic imaging systems may output high definition video images to the surgeon, and may allow features such as zoom to provide a “magnified” view that allows the surgeon to identify specific tissue types and characteristics, as well as to work with increased precision. In a typical surgical field, however, the quality of the image captured by a camera in the electronic stereoscopic imaging system is limited by the signal-to-noise ratio of the camera.
As a camera collects light, the captured light is converted to electrons and stored in wells of an image sensor. There is one well per pixel. FIG. 1 is a schematic illustration of a well 101 for a red pixel R, a well 102 for a green pixel G, and a well 103 for a blue pixel B. As a camera collects more electrons into its wells, the signal grows while the noise stays relatively constant, and so the signal-to-noise ratio increases, i.e. the signal captured in the well increases with respect to the noise.
A physical property of light capture by a camera is that the more light a camera pixel captures, the better the camera can estimate the rate at which the light was captured. However, if a camera pixel collects too much light and overfills a well, the signal for that pixel is lost and no longer valid. Therefore, an exposure time of the camera is set to try and collect light to fill all of its electron wells 101, 102, 103 as high as possible without overfilling any one well.
In a typical surgical site scene that is illuminated by white light for general observations, red is the predominant color in the scene captured by a camera. This because most of the reflected light is in the red spectrum relative the blue and green spectrums.
Typically, a color video camera used in a teleoperated surgical system includes a color filter array. The color filter array is a mosaic of different colored filters. Ideally, each different color filter passes only a portion of the visible electromagnetic spectrum corresponding to the spectrum of a particular color, e.g., a first set of filters in the color filter array passes primarily red light, a second set of filters pass primarily green light, and a third set of filters pass primarily blue light.
The camera includes an image sensor that includes pixels that capture the light that passes through the color filter array. Each pixel is a well that fills up with electrons as the light is captured. The set of pixels in the camera that capture the light that passes through the first set of filters are included in a first color channel of the camera. The set of pixels in the camera that capture the light that passes through the second set of filters are included in a second color channel of the camera. The set of pixels in the camera that capture the light that passes through the third set of filters are included in a third color channel of the camera.
As is known to those knowledgeable in the field, in one example, white light illumination is made up of a combination of red spectrum light, green spectrum light and blue spectrum light that looks white to the eyes of a human with normal color perception. However, due to the predominant reflection of the red spectrum light by the surgical site, red pixel well 101 (FIG. 1) typically fills up much faster than either green pixel well 102, or blue pixel well 103. To prevent red pixel well 101 from overflowing, the exposure of the camera is set to limit the light collected so that red pixel well 101 does not overflow.
The consequence of stopping the collection of light when the wells of the color channel receiving the most light are about to overflow is that the wells of the other color channels may not be full as illustrated in FIG. 1. In the example of FIG. 1, green well 102 and blue well 103 are less than fifty-percent full when the collection of light is stopped. The signal-to-noise ratio of these less-full color channels is significantly less than the signal-to-noise ratio of the color channel or channels that were about to overflow. Again, for the example of FIG. 1, the signal-to-noise ratio of the red channel is about six, while the signal-to-noise ratio of each of the green and blue channels is about three.
A camera has worse signal-to-noise ratio performance when not all of wells 101, 102, 103 of the camera color channels are full. The signals from less full wells 102 and 103 must have a gain applied to the signals as part of a white balance stage in the surgical system's image processing to create an image for display. White balancing is necessary to ensure that when a camera captures an image of a white surface, the white surface appears white on the display monitor. White balancing consists of amplifying less-full color channels (the blue and green color channels in FIG. 1), e.g., applying a digital gain, such that that all the color channels have equal values when the camera captures an image of a white surface. The amplification of these less-full well signals increases the noise of these color signals relative to the other color signals, which further increases the noise in the final image.